Papers with Chinese NLU tasks
RoCBert: Robust Chinese Bert with Multimodal Contrastive Pretraining (2022.acl-long)
Copied to clipboard
| Challenge: | Large-scale pretrained language models have achieved SOTA results on NLP tasks but are vulnerable to adversarial attacks especially for logographic languages like Chinese. |
| Approach: | They propose a pretrained Chinese Bert that is robust to various forms of adversarial attacks like word perturbation, synonyms, typos, etc. |
| Outcome: | The proposed model outperforms baselines on 5 Chinese NLU tasks without sacrificing performance on clean testsets. |